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Managing  Complexity 


Managers  are  responsible  for  overseeing  increasingly  complex  projects, 
programs,  and  operational  processes. 

•  Multiple  points  of  management  control 

•  Complex  tasks 

•  Complex,  distributed  support 
technologies 

•  Multiple,  detailed  status  reports 

•  A  variety  of  management 
techniques  (project,  security, 
financial,  technology,  etc.) 

•  Requirements  of  multiple  stakeholders 


— Assuring  Mission  Success  in  Complex  Settings 

Software  Engineering  Institute  CamegieMellon  Alberts  and  Dorofee,  March  2007  3 

©  2006  Carnegie  Mellon  University 


Need  for  a  New  Approach 


Traditional  analysis  and  management 
approaches  not  designed  for  complex 
environments 

•  Cannot  handle  organizational  and 
technological  complexity 

•  Do  not  easily  scale  to  distributed 
environments 

Need  new  methods,  tools,  and 
techniques  to 

•  Position  projects,  programs,  and 
processes  for  success 

•  Establish  and  maintain  confidence  in 
achieving  objectives 
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Managing  for  Mission  Success 


Managing  for  mission  success  requires  establishing  and  maintaining  a 
reasonable  degree  of  confidence  that  a  mission’s  objectives  will  be 
successfully  achieved. 


Software  and  System 
Development 
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SEI  MOSAIC: 

Managing  for  Success 
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Overview 


SEI  Mission-Oriented  Success  Analysis  and  Improvement  Criteria 
(MOSAIC)  is  a  structured  decision-making  approach  that 

•  Establishes  a  reasonable  degree 
of  confidence  in  the  potential  for 
a  successful  mission 

•  Helps  ensure  mission  success 
in  projects,  programs, 
processes,  and  systems 
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Strategic  Allocation  of  Resources 


People  need  a  way  to  make  appropriate  tradeoffs  among  a  broad 

range  of  factors. 
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SEI  MOSAIC:  A  Lifecycle  Approach 


Perform  during  any  lifecycle  phase 

Supports  most  system  lifecycle 
models 


Strategy  Design  Testing/  Operations/ 

Evaluation  Planning  Integration  Maintenance 


4 
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Concept  Requirements  Development  Release/ 

Exploration  Analysis  Activities  Production 
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Managing  the  Outcome 


An  outcome  is  the  result  achieved  when 
executing  a  mission. 

•  A  range  of  potential  outcomes  is  possible 

•  Some  outcomes  are  acceptable — success 

•  Some  outcomes  are  unacceptable — failure 

SEI  MOSAIC  defines  an  approach  for 
managing  the  expected  outcome  in  relation 
to  the  desired  outcome. 

•  What  is  the  mission  likely  to  achieve? 

•  What  do  I  want  the  mission  to  achieve? 
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Range  of  Potential  Outcomes 


Current  Conditions 


Range  of  Potential 
Outcomes 


Potential  Events 
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Positioning  for  Success 


A  range  of  outcomes  is  possible  for  any  given 
mission. 

Conditions  and  potential  events 

•  affect  mission  execution  and  influence  a 
mission’s  eventual  outcome 

•  must  be  appropriately  managed  to  position 
a  mission  for  success 

The  objective  is  to  drive  the  expected 
outcome  toward  acceptable  states. 


Acceptable 

Outcomes 


Unacceptable 

Outcomes 
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Unique  Features  of  SEI  MOSAIC 


Traditional  Risk  Management 

SEI  MOSAIC 

Narrow  scope 

(single  project,  system,  or  organization) 

Linear  view  of  risk 

(cause-effect  pairs) 

Threat-driven 

Hazard  avoidance 

“Playing  not  to  lose” 

Broad  scope 

(distributed  processes,  systems  of  systems) 

Interrelated  view  of  risk 

Outcome-driven 

Opportunity  seeking 

“Playing  to  win” 
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SEI  MOSAIC  Project 
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Characteristics  of  Current  Approaches 


A  prevalence  of  one-size-fits-all  analysis  and  management  methods 

•  Complex  solutions  that  are  not  easily  tailored  (especially  to  small 
organizations) 

•  Tied  to  specific  domains  or  problems 

Locally  optimized  results 

•  Narrow  tradeoff  space 

•  Subset  of  the  lifecycle 

•  Narrow  scope  (e.g.,  single  project,  system,  or  organization) 
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SEI  MOSAIC  Approach 


Each  SEI  MOSAIC  method  is  tailored  to 

•  A  given  situation,  problem  space,  or  lifecycle  phase 

•  The  domain  or  application  area 

•  The  circumstances  at  hand 

SEI  MOSAIC  is  focused  on  global  effectiveness  and  mission  success. 

•  Broad  tradeoff  space 

•  Lifecycle  focus  (development  and  operations) 

•  Broad  scope  (e.g.,  distributed  processes,  supply  chains,  systems 
of  systems) 
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SEI  MOSAIC  Toolkit 
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SEI  MOSAIC  Methods 


Our  current  work  is  focused  on  developing  a  suite  of  analysis  methods. 


Two  methods  so  far: 

•  Mission  Diagnostic  is  a  basic  approach 
that  provides  a  quick,  high-level 
evaluation. 

•  Mission  Assurance  Analysis  Protocol 
(MAAP)  is  a  comprehensive  approach 
that  provides  an  in-depth  evaluation. 
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Mission  Diagnostic 


What 

A  time-efficient  means  of  assessing 
the  potential  for  success 

Why 

To  determine  whether  conditions  are 
favorable  for  a  successful  outcome 

Key  Results 

An  evaluation  of  key  indicators  and  an 
estimate  of  the  success  potential 
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Key  Indicators 


Evaluate  a  set  of  indicators 
representing  key  aspects  of 
management,  for  example: 

•  Realistic  goals 

•  Customer  requirements 

•  Staffing  requirements 

•  Technology  feasibility 

•  Plans  and  schedules 


“Are  customer  requirements  and  needs  well  understood ?” 
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Evaluating  Key  Indicators 


Question 

No 

Likely 

no 

Answer 

Equally 

likely 

Likely 

yes 

Yes 

Are  goals  realistic  and 
well  articulated? 

q 

q 

q 

n 

q 

Each  indicator  is  evaluated  based  on  the  data  that  have 
been  collected. 

Uncertainty  is  incorporated  into  the  range  of  answers  for 
each  indicator. 
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Indicator  Evaluation  Criteria 


Answer 

Definition 

Yes 

The  answer  is  almost  certainly  “yes.”  Very  little  uncertainty 
exists. 

Likely  yes 

The  answer  is  most  likely  “yes.”  However,  a  degree  of 

Equally  likely 

The  answer  is  just  as  likely  to  be  “yes”  or  “no.”  A  high 
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No 
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Indicator  Analysis 
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A  simple  analysis  provides  insight  into 
a  mission’s  health. 
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Managing  the  Potential  for  Success 


The  goal  is  to  improve  a  mission’s  current  state  of  health. 
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Indicators  for  Software  Development  Programs 


•  Are  goals  realistic  and  well 
articulated? 

•  Are  communication  and 
information  sharing  about 
mission  activities  effective? 

•  Are  customer  requirements  and 
needs  well  understood? 

•  Are  stakeholder  politics  or  other 
external  pressures  minimal? 

•  Does  the  process  design 
support  efficient  and  effective 
execution? 


•  Are  process  control 
mechanisms  are  effective? 

Is  task  execution  efficient  and 
effective? 

•  Are  staffing  and  funding 
sufficient  to  execute  all  mission 
activities? 

•  Are  the  technological  and 
physical  infrastructures 
adequate  to  support  all  mission 
activities? 

•  Are  changing  circumstances 
and  unpredictable  events 
effectively  managed? 
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Evaluating  Indicators 


The  following  data  are  recorded  for 
each  indicator: 

•  Indicator  score 

•  Rationale  for  indicator  score 

•  Analysis  approach 
(for  example,  intuition, 
qualitative  analysis, 
quantitative  analysis,  other) 

•  Potential  actions 

•  Evaluators 

•  Date 
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Mission  Diagnostic  Exercise  and  Handout 
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Tailoring  Questions 


The  following  questions  can  be  used  when  tailoring  or  developing  a  set 
of  indicators: 

•  What  constitutes  a  successful  result  for  the  project  or  process? 

•  What  constitutes  an  unsuccessful  result,  or  failure,  for  the  project 
or  process? 

•  What  circumstances  or  conditions  tend  to  produce  a  successful 
outcome  when  conducting  the  project  or  process? 

•  What  circumstances  or  conditions  tend  to  produce  an 
unsuccessful  outcome,  or  failure,  when  conducting  the  project  or 
process? 
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Mission  Diagnostic  Across  the  Lifecycle 


Strategy  Design  Testing/  Operations/ 

Evaluation  Planning  Integration  Maintenance 


4 


Concept  Requirements  Development  Release/ 

Exploration  Analysis  Activities  Production 


How  much  uncertainty  in  these  indicators  can  you  tolerate  at  different 
points  in  the  lifecycle? 
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MAAP 


What 

A  systematic  approach  for  thoroughly 
analyzing  the  potential  for  success 

Why 

To  characterize  the  full  range  of  drivers 
affecting  the  success  potential 

To  set  management  priorities  to  ensure 
the  success  potential  is  maintained 
within  tolerance 


Key  Results 

An  operational  model,  customized  analysis  artifacts,  a  measure  of  the 
success  potential,  and  strategies  for  keeping  the  success  potential  within 
tolerance 
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Drivers  of  Success  and  Failure 


A  broad  range  of  drivers 
must  be  considered  when 
analyzing  the  potential  for 
mission  success. 
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Mission 


A  mission  threat  is  a  fundamental  flaw,  or  weaknesses,  in 
the  purpose  and  scope  of  a  work  process. 
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Process  Design 


A  design  threat  is  an  inherent  weakness  in  the  layout  of  a 
work  process. 
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Activity  Management 


An  activity  threat  is  a  flaw,  or  weaknesses,  arising  from  the 
manner  in  which  activities  are  managed  and  performed. 
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Operational  Environment 


An  environment  threat  is  an  inherent  constraint,  weakness, 
or  flaw  in  the  overarching  operational  environment  in  which  a 
process  is  conducted. 
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Event  Management 


by  an  unpredictable  occurrence  that  introduces 
unexpected  change  into  a  process. 
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Scenario-Based  Analysis 


Scenario  1 

Expected  operational 
conditions 


Scenario  2 
When  stressed  by 
Event  1 


Event  1 


Scenario  3 
When  stressed  by 
Event  2 


Outcome  during 
expected  operational 
conditions 


Expected 

outcome 
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Complex  Risks 


Outcome  Analysis 


The  goal  is  to  ensure  that  the  expected  outcome  for  each  objective  in 
all  evaluated  scenarios  is  acceptable  to  key  stakeholders. 


Best  Outcome 


Worst  Outcome 
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>  Gap  between  current  and 
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Unique  Features  of  SEI  MOSAIC 


•  Manages  the  potential  for  success 

•  Can  be  applied  to  highly  distributed 
programs  and  operational 
processes 


Provides  a  ‘global’  view  of  a  mission 


Analyzes  issues  that  are  too 
complex  for  other  techniques 
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Potential  Application  Areas 


•  Large,  distributed  software  development  programs 

•  Organizations  in  dynamic,  rapidly  changing  business  environments 

•  Organizations  with  strict  reliability,  security,  and  safety  requirements 

•  Large,  distributed  supply  chains 

•  Processes  supporting  critical  infrastructures 

•  Distributed  information-technology  (IT)  processes 


Future  Research  and  Development 


Refine  the  current  SEI  MOSAIC  analysis 
protocols. 

Define  and  pilot  additional  SEI  MOSAIC 
analysis  protocols. 

Begin  work  on  an  approach  for  real-time 
monitoring  and  management  of  mission 
outcomes. 
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For  Additional  Information 


Telephone 

412/268-5800 

Fax 

412/268-5758 

WWW 

http://www.sei.cmu.edu/msce/ 

U.S.  mail 

Customer  Relations 

Software  Engineering  Institute 

Carnegie  Mellon  University 

Pittsburgh,  PA  15213-3890 
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